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Abstract 

Definable zero-sum stochastic games involve a finite number of states and action sets, 
reward and transition functions that are definable in an o-minimal structure. Prominent 
examples of such games are finite, semi-algebraic or globally subanalytic stochastic games. 
We prove that the Shapley operator of any definable stochastic game with separable transition 
and reward functions is definable in the same structure. This result is used to prove that 
any separable definable game has a uniform value; in the case of polynomially bounded 
structures we also provide convergence rates. Using an approximation procedure, we actually 
establish that general zero-sum games with separable definable transition functions have a 
uniform value. These results cover the particular case of stochastic games with polynomial 
transitions, definable games with finite actions on one side, definable games with perfect 
information or switching controls, as well as nonlinear maps arising in risk sensitive control 
and Pcrron-Frobenius theory 

Keywords Zero-sum stochastic games, Shapley operator, o-minimal structures, definable games, 
uniform value, nonexpansive mappings, nonlinear Perron-Frobenius theory, risk-sensitive control. 

1 Introduction 

Zero-sum stochastic games have been widely studied since their introduction by Shapley |40j in 
1953 (see the textbooks |42| [Po| 127} I3T] for an overview of the topic). They model long term 
interactions between two players with completely opposite interest; they appear in a wealth of 
domains including computer science, population dynamics or economics. In such games the 
players face, at each time n, a zero-sum game whose data are determined by the state of nature. 
The evolution of the game is governed by a stochastic process which is partially controlled by 
both players through their actions, and which determines, at each stage of the game, the state 
of nature and thus the current game faced by both players. The players aim at optimizing their 
gain over time which implies, depending on the mathematical model, specific choices of payoff 
evaluations. 

We shall focus here on two kind of payoff evaluations which are based on Cesaro and Abel 
means. For any finite horizon time n, one defines the "repeated game" in n stages for which 
each player aims at optimizing his averaged gain. Similarly for any discount rate A, one defines 
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the A-discounted game for infinite horizon games. Under minimal assumptions these games have 
values, and an important issue in Dynamic Games theory is the asymptotic study of these values 
(see Subsection 13. ip . These aspects have been dealt along two lines: 

— The "asymptotic approach" consists in the study of the convergence of these values when 
the players grow more and more patient - that is when n goes to infinity or A goes to 0. 

— The "uniform value approach", for which one seeks to establish that, in addition, both 
players have near optimal strategies that do not depend on the horizon (provided that the 
game is played is long enough). 

The asymptotic approach is less demanding as there are games |49] with no uniform value 
but for which the value does converge to a common limit; the reader is referred to |27| for a 
thorough discussion on those two approaches and their differences in zero-sum repeated games. 

For the asymptotic approach, the first positive results were initially obtained in recursive 
games [15] . games with incomplete information [3j[28] and absorbing games [21] . In 1976, Bewley 
and Kohlberg settled, in a fundamental paper the case of games with finite sets of states 
and actions. Their proof is based on the use of Puiseux Lemma for the discounted values curve. 
Several types of improvements based on techniques of semi-algebraic geometry were developed 
in [30JI22]- 

Bewley-Kohlberg's result of convergence was later considerably strengthened by Mertens and 
Neyman who proved |25| the existence of a uniform value in this finite framework, when the 
payoff is observed by the players. 

The semi-algebraic techniques used in the proof of Bewley and Kohlberg have long been 
considered as specifically related to the finiteness of the action sets and it seemed that they could 
not be adapted to wider settings. In [39j the authors consider a special instance of polynomial 
games but their focus is computational and concerns mainly the estimation of discounted values 
for a fixed discount rate. In order to go beyond their result and to tackle more complex games, 
most researchers have used topological or analytical arguments, see e.g. |26| 133"! l35| |36j [371 S3 
|44"1 145] , The common feature of most of these papers is to study the analytical properties of the 
so-called Shapley operator of the game in order to infer various convergence results of the values. 
This protocol, called the "operator approach" by Rosenberg and Sorin, grounds on Shapley 's 
theorem which ensures that the dynamic structure of the game is entirely represented by the 
Shapley operator. 

Our paper can be viewed as a 'definable operator approach" . In the spirit of Bewley- Kohlberg 
and Neyman, we identify first a class of potentially "well-behaved games" through their under- 
lying geometric features (definable stochastic games) and we show how these properties actually 
impact the structure of the Shapley operator (its definability and subsequent properties). By 
the use of Mertens-Neyman result this implies in turn the existence of a uniform value for a wide 
range of games (e.g. polynomial games). 

Before giving a more precise account of our results, let us describe briefly the topological/geo- 
metrical framework used in this paper. The rather recent introduction of o-minimal structures 
as models for a tame topology (see |13j ) is a major source of inspiration to this work. O- 
minimal structures can be thought of as an abstraction of semi- algebraic geometry through an 
axiomatization of its most essential properties. An o-minimal structure consists indeed in a 
collection of subsets belonging to spaces of the form M n , where n ranges over N, called definable 
sets (0). Among other things, this collection is required to be stable by linear projections and 

1 Functions are called definable whenever their graph is definable. 
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its 'one-dimensional" sets must be finite unions of intervals. Definable sets are then shown to 
share most of the qualitative properties of semi-algebraic sets like finiteness of the number of 
connected components or differential regularity up to stratification. 

Our motivation for studying stochastic games in this framework is double. First, it appears 
that definability allows one to avoid highly oscillatory phenomena in a wide variety of settings: 
partial differential equations |41] . Hamilton- Jacobi-Bellman equations and control theory (see 
|46] and references therein), continuous optimization |20| . We strongly suspect that definability 
is a simple means to ensure the existence of a value to stochastic games. 

Another very important motivation for working within these structures is their omnipresence 
in finite-dimensional models and applications (see e.g. [20] and the last section). 

The aim of this article is therefore to consider stochastic games -with a strong focus on their 
asymptotic properties- in this o-minimal framework. We always assume that the set of states is 
finite and we say that a stochastic game is definable in some o-minimal structure if all its data 
(action sets, payoff and transition functions) are definable in this structure. The central issue 
behind this work is probably: 

(Q) Do definable stochastic games have definable Shapley operators ? 

As we shall see this question plays a pivotal role in the study of stochastic games. It seems 
however difficult to solve it in its full generality and we are only able to give here partial results. 
We prove in particular that any stochastic game with definable, separable reward and transition 
functions (e.g. polynomial games) yields a Shapley operator which is definable in the same struc- 
ture. The separability assumption is important to ensure definability in the same structure, we 
indeed describe a rather simple semi-algebraic game whose Shapley operator is globally suban- 
alytic but not semi-algebraic. The general question of knowing whether a definable game has a 
Shapley operator definable in a possibly larger structure remains fully open. 

An important consequence of the definability of the Shapley operator is the existence of a 
uniform value for the corresponding game (Theorem [3j • The proof of this result is both based 
on the techniques and results of [30] and |25] , For games having a Shapley operator definable in 
a polynomially bounded structure, we also show, in the spirit of Milman [29], that the rate of 
convergence is of the form O(^) for some positive 7. 

These results are used in turn to study games with arbitrary continuous reward functions 
(not necessarily definable) and separable and definable transition functions. Using the Stone- 
Weierstrass and Mertens-Neyman theorems, we indeed establish that such games have a uniform 
value (Theorem [7J- This considerably generalizes previous results; for instance, our central 
results imply that: 

— definable games in which one player has finitely many actions, 

— games with polynomial transition functions, 

— games with perfect information and definable transition functions, 

— games with switching control and definable transition functions, 
have a uniform value. 

The above results evidence that most of the asymptotic complexity of a stochastic game 
lies in its dynamics, i.e. in its transition function. In the same spirit a recent companion work 
|47] shows, through a counterexample to the convergence of values, that the o-minimality of 
the underlying stochastic process is a fundamental assumption. The example involves finitely 
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many states, simple compact action sets, and continuous transition and payoffs but the transition 
functions are typically non definable since they oscillate infinitely many times on a compact set. 

We also include an application to a class of maps arising in risk sensitive control |17| |9j [2] 
and in nonlinear Perron- Frobenius theory (growth minimization in population dynamics). In 
this context, one considers a self-map T of the interior of the standard positive cone of M d , and 
looks for conditions of existence of the geometric growth rate [T fe (e)]* as k — > oo, where e is 
an arbitrary vector in the interior of this cone. This leads to examples of Shapley operators, the 
conjugates of T by log-glasses, that are definable in the log-exp structure. 

The paper is structured as follows. The first sections give a basic primer on the theory of 
o-minimal structures and on stochastic games. We introduce in particular definable zero-sum 
stochastic games and discuss several subclasses of games. The main result of that section is the 
following: if the Shapley operator of a game is definable in an o-minimal structure, this game has 
a uniform value. Since the Shapley operator is itself a one-shot game where the expectation of the 
future payoffs acts as a parameter, we study one-shot parametric games in Section [4] We prove 
that the value of a parametric definable game is itself definable in two cases: either if the game 
is separable, or if the payoff is convex. These results are in turn used in Section [5] to prove the 
existence of a uniform value for several classes of games including separably definable games. We 
finally point an application to a class of "log-exp" maps arising in population dynamics (growth 
minimization problems) and in risk sensitive control. 

2 O-minimal structures 

O-minimal structures play a fundamental role in this paper; we recall here the basic results 
that we shall use throughout the article. Some references on the subject are van der Dries |13j . 
van der Dries-Miller [JJ], Coste [XT] . 

For a given p in N, the collection of subsets of W is denoted by V(W). 

Definition 1 (o-minimal structure, Definition 1.5]). An o-minimal structure on (M, +,.) is 
a sequence of Boolean algebras O = (O p ) pg N with O p C V(M P ), such that for each p£N: 

(i) if A belongs to O p , then A x K and M x A belong to O p+ \ ; 

(ii) if II : MP +1 — > MP is the canonical projection onto MP then for any A in O p +i, the set H(A) 
belongs to O p ; 

(iii) Op contains the family of real algebraic subsets of MP, that is, every set of the form 

{x € M p : g(x) = 0}, 

where g : MP — > M is a real polynomial function ; 

(iv) the elements of 0\ are exactly the finite unions of intervals. 

A subset of MP which belongs to an o-minimal structure O, is said to be definable in O or simply 
definable. A mapping F : S C M p ->■ M q is called definable (in O), if its graph {(x, y) G M p x M q : 
y S F{x)} is definable (in O) as a subset of M p x M q . Similarly if g : M p — > ( — oo,+oo] (resp. 
g : M p — > [— oo, +oo)) is a real-extended-valued function, it is called definable (in O), if its graph 
{(x,r) £l p xl: g(x) = r} is definable (in O). 
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Remark 1. The smallest o-minimal structure is given by the class SA of real semi- algebraic 
objects (d). We recall that a set A C W is called semi-algebraic if it can be written as 



I k 

A = \J fl {x G RP : = 0, /^-(x) < 0}, 

i=i i=l 

where the gij,hij : W p — > R are real polynomial functions on MP. The fact that SA is an o- 
minimal structure stems from the Tarski-Seidenberg principle (see j7]) which asserts the validity 
of item (ii) in this class. 

The following result is an elementary but fundamental consequence of the definition. 

Proposition 1 ([Ii]). Let AdW and g : A W be definable objects. 

(i) Let B C A a definable set. Then g(B) is definable. 

(ii) Let C C R 9 be a definable set. Then y^^C) is definable. 

One can already guess from the above definition and proposition that definable sets behave 
qualitatively as semi- algebraic sets. The reader is referred to |14|ITT] for a comprehensive account 
on the topic. 

Example 1 (max and min functions). In order to illustrate these stability properties, let us 
consider nonempty subsets A, B of R p , R 9 respectively, and <j:ixB->Ma definable function. 
Using the projection axiom, we see that both A and B are definable. Set h{x) = infygs g(x, y) 
for all x in A and let us establish the definability of h; note that the domain of h, i.e. dom/i = 
{x £ A : h(x) > — oo} may be smaller than A and possibly empty. The graph of h is given by 

graph h := {(x,r) G A x R : (Vy G B,g(x,y) ^ r) and (Ve > 0, By G B,g(x,y) < r + e)} . 

As explained below, the assertion 

(Vy G B, g(x, y) ^ r) and ( Ve > 0, 3y G #(x, y) < r + e) , 

is called a first order formula, but the main point for the moment is to prove that such a formula 
necessarily describes a definable set. 
Consider the sets 

T = {(x, r) G A x R : Ve > 0, By G B, g(x, y) < r + e} , 

So = {(x, y, r, e) G A x 1? x R x (0, +oo) : g(x, y) - r - e < 0} . 

We wish to prove that T is definable. Projecting So via n(x, y, r, e) = (x,r, e), one obtains the 
definable set Si = {(x,r, e) G A x R x (0, +oo) : 3y G B,g(x,y) — r — e < 0}. Introducing 
n'(x, r, e) = (x, r), we see that T can be expressed as 

{A x R) \ IT [(ixKx (0, +oo)) \ Si] . 

Since the complement operations preserve definability, T is definable. Using this type of idea 
and Definition [TJ we can prove similarly that 

T' = {(x, r) G A x R : Vy G B, y(x, y) ^ r} 

is definable. Hence graph h = T fl T' is definable and thus /i is definable. 

2 This is due to axiom (iii). Sometimes this axiom is weakened |13] , allowing smaller classes than SA, for 
instance the structure of semilinear sets. 
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The most common method to establish the definability of a set is thus to interpret it as 
the result of a finite sequence of basic operations on definable sets (projection, complement, 
intersection, union). This idea is conveniently captured by the notion of a first order formula. 
First order formulas are built inductively according to the following rules: 

— If A is a definable set, x G A is a first order formula 

— If P(xi, . . . , x p ) and Q(x\, . . . , x q ) are first order formulas then (non P), (P and Q), and 
(P or Q) are first order formulas. 

— Let A be a definable subset of R p and P(x±, . . . , x p , y%, . . . , y g ) a first order formula then 
both 

(3xeA,P(x,y)) 
(VxeA,P(x,y)) 

are first order formulas. 

Note that Proposition [1] ensures that il g(xi , . . . , x p ) = 0" or l g(x\ , . . . , x p ) < 0" are first order 
formulas whenever g : W — > R is definable (e.g. polynomial). 
It is then easy to check, by induction, that: 

Proposition 2 ( [1 lj ) . If &(x±, . . . ,x p ) is a first order formula, then {(xi, . . . , x p ) G MP : 
<&(xi, . . . , x p )} is a definable set. 

Remark 2. A rigorous treatment of these aspects of o-minimality can be found in |23| . 

An easy consequence of the above proposition that we shall use repeatedly and in various 
form is the following. 

Proposition 3. Let f2 be a definable open subset o/R n and g : Q — > R m a definable differ entiable 
mapping. Then its derivative g' is definable. 

There exists many regularity results for definable sets |14] , In this paper, we essentially use 
the following fundamental lemma. 

Let O be an o-minimal structure on (R, + , .). 

Monotonicity Lemma ([U Theorem 4.1]). Let / :/ cM->Rfcd definable function and 
k G N. Then there exists a finite partition of I into I disjoint intervals Ji,...,ij such that f 
restricted to each nontrivial interval Ij, j € {1,...,/} is C k and either strictly monotone or 
constant. 

We end this section by giving examples of o-minimal structures (see [14] and references therein). 
Examples (a) (globally subanalytic sets) There exists an o-minimal structure, that contains 
all sets of the form {(x,t) G [-1, If x R : f(x) = t} where / : [-1, If ->• R (p G N) is an 
analytic function that can be extended analytically on a neighborhood of the box [—1, l] p . The 
sets belonging to this structure are called globally subanalytic sets; see |14j and also [5] for an 
account on subanalytic geometry. 
For instance the functions 

sin : [—a, a] — > R 

(where a ranges over R + ) are globally subanalytic, while sin : R — > R is not (else the set 
sin~ 1 ({0}) would be finite by Proposition [1] and Definition [TJiv)). 

(b) (log-exp structure) There exists an o-minimal structure containing the globally subanalytic 
sets and the graph of exp : R — > R. 

We shall also use a more "quantitative" characteristic of o-minimal structures. 



Definition 2 (Polynomially bounded structures). An o-minimal structure is called polynomially 
bounded if for all function ip : (a, +00) — > R there exists a positive constant C and an integer N 
such that \ip(t)\ ^ Ct N for all t sufficiently large 

The classes of semi-algebraic sets or of globally subanalytic sets are polynomially bounded 
|14| . while the log-exp structure is obviously not. 

We have the following result in the spirit of the classical Puiseux development of semi- 
algebraic mappings. 

Corollary 1 (|14|). If e > and : (0, e) — > R is definable in a polynomially bounded o-minimal 
structure there exist c £ R and a £ R such that 



3 Stochastic games 

3.1 Definitions and fundamental properties 

Stochastic games: definition. A stochastic game is determined by 

— Three sets: a finite set of states Q, with cardinality d, and two nonempty sets of actions 
IcPandFc R q . 

— A payoff function g : !)x!x7->R and a transition probability p : f2 x X x Y — > A(f2), 
where A(f2) is the set of probabilities over fi. 

Such a game is denoted by (£1, X,Y, g, p). Unless explicitly specified, we will always assume the 
following, which guarantees that the finite horizon and discounted values do exist. 



Strategies and values. The game is played as follows. At time n = 1, the state oj\ is known 
by both players, player 1 (resp. 2) makes a move x in X (resp. y in Y) and the resulting payoff 
gi := g(xi,yx,0Jx) is observed by the two players. The new state 0J2 is drawn according to the 
probability distribution y±, U){), both players observe this new state and can thus play 

accordingly. This process goes on indefinitely and generates a stream of actions Xi,yi, states Wj 
and payoffs gi = g(xi,yi,Ui). Denote by H n = {X x Y x Q) n x 0, the sets of stories of length n, 
H = U n ^H n the set of all finite stories and = (Q x X x Y) N the set of infinite stories. A 
strategy for player 1 (resp. player 2) is a mapping 



A triple (a, r, UJ\) defines a probability measure on whose expectation is denoted E c 
The stream of payoffs corresponding to the triple (a, t,uj{) can be evaluated, at time n, as 



<P(t) =ct a +o(t a ), iG(0,e). 



Standing assumptions (A): The reward function g and the transi- 
tion function p are continuous; both action sets X, Y are nonempty 
compact sets. 



a :H -> A(X) (resp. r : H -»• A(Y)). 




(3.1) 



7 



The corresponding game is denoted by T n ; Assumption (A) allows us to apply Sion's Theo- 
rem |421 Theorem A. 7, p. 156], which shows that this game has a value v n {u)\) or simply (v n )i. 
When the sequence v n = ((v n )i, . . . , (v n )d) converges as n tends to infinity the game is said to 
have a value. 

Another possibility for evaluating the stream of payoffs is to rely on a discount factor A e]0, 1[ 
and to consider the game T\ with payoff 

lx (a, r, wi) = E m ^A - A)'" 1 ^ • (3.2) 

Applying once more Sion result this game has a value which we denote by v\(lui) or simply (v\)i. 
The vector v\ is defined as v\ = ((v\)i, ■ ■ ■ , (v\)d)- One of the central question of this paper is 
to find sufficient conditions to have 

lim v n = lim v\. 

n-t+oa A— >0, A>0 

Shapley operator and Shapley's theorem. Let us now describe the fundamental result of 
Shapley which provides an interpretation of the value of the games T n as rescaled iterates of a 
nonexpansive mapping. In the same spirit, the discounted values V\ appear as fixed points of a 
family of contractions. 

Let (Q,X,Y,g,p) be an arbitrary stochastic game. The Shapley operator associated to such 
a game is a mapping ^ : R d — > M. d , whose fcth component is defined through 



*Jfe(/l, ■■■ ,fd) = max min / / 



g(x,y,uj k ) +^2p(uji\x,y,uj k )fi 
i=l 



dp(x)dv{y). (3.3) 



Observe as before, that the maximum and the minimum can be interchanged in the above 
formula. The space R d can be thought of as the set of value functions J-~({1, . . . , d}; R), i.e. the 
functions which map {1, . . . , d} ~ f2 (set of states) to R (real-space of values). It is known that 
a self-map ^ of R rf can be represented as the Shapley operator of some stochastic game — that 
does not satisfy necessarily assumption (A) - if and only if it preserves the standard partial order 
of R d and commutes with the addition of a constant |22] , Moreover, the transition probabilities 
can be even required to be degenerate (deterministic) , see |38| IT9] . 



Theorem 2 (Shapley, [40]). 

(i) For every positive integer n, the value v n of the game Y n satisfies v n = ^^ n (0). 

(ii) The value v\ of the discounted game T\ is characterized by the following fixed point condition 

v x = \*(^^v x ). (3.4) 

Uniform value. A stochastic game is said to have a uniform value Voo if both players can almost 
guarantee Voq provided that the length of the n-stage game is large enough. Formally, Voq is the 
uniform value of the game if for any e > 0, there is a couple of strategies of each player (cr, r) 
and a time N such that, for every n ^ N, every starting state oj\ and every strategies a 1 and r', 

7„(c7,r / ,wi) ^ Uoo(wi)-e 
7 n (c7 / ,T,wi) < Uoo(wi) + e 

It is straightforward to establish that if a game has a uniform value ^oo, then v n and v\ 
converges to v^. The converse is not true however, as there are games with no uniform value 
but for which v n and V\ converge |28j . 
Some subclasses of stochastic games. 
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— Markov Decision Processes : they correspond to one-player stochastic games (the choice of 
Player 2 has no influence on payoff nor transition). In this case the Shapley operator has 
the particular form 



*ifc(/l, •••,/<*) = max 

xex 



g{x,u) k ) + ^2p(ui\x,uj k )fi 



8=1 



(3.5) 



for every k = 1, . . . , d. 



— Games with perfect information : each state is entirely controlled by one of the player (i.e. 
the action of the other player has no influence on the payoff in this state nor on transitions 
from this state). In that case, the Shapley operator has a specific form : for any state u; k 
controlled by Player 1, 



*fc(/l,- ■ ■ ,fd) =max 

x&X 



g{x,uj k ) + ^2p(uJi\x,uj k )fi 
i=i 

and for any state oj k controlled by Player 2, 



(3.6) 



*fe(/l, ■■■Jd)= min 



g(y,u k ) + ^2p(ui\y,u k )fi 



i=l 



(3.7) 



Games with switching control : in each state the transition is entirely controlled by one 
of the player (i.e. the action of the other player has no influence on transitions from this 
state, but it may alter the payoff). In that case, the Shapley operator has a specific form: 
for any state u: k where the transition is controlled by Player 1, 



Vk(fl,---,fd)= max / 



mmg(x,y,u k ) + y~)p(ui\x,u k )fi 

1=1 



dp(x), (3i 



and for any state oj k where the transition is controlled by Player 2, 

d 



= min / 

v&A(Y) J x 



max g(x,y,u} k ) + y~]p(ui\y,Uk)fi 
igX 1 — » 



i=i 



dv{y). (3.9) 



Remark 3. Recall that we made assumption (A) in order to prove the existence of v\ and v n . 
For Markov decision processes and games with perfect information this existence is automatic 
as soon as the payoff is bounded, so there is no need to assume continuity of g or p. 

Definable stochastic games. Let O be an o-minimal structure. A stochastic game is called 
definable if both the payoff function and the probability transition are definable functions. 

Observe in the above definition that the definability of g implies that the action sets are 
also definable. Note also that the space A(f2), is naturally identified to the d simplex and is 
thus a semi- algebraic set. Hence there is no possible ambiguity when we assume that transition 
functions are definable. 

The questions we shall address in the sequel revolve around the following two ones 
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(a) If a Shapley operator of a game is definable, what are the consequences in terms of games 
values? 

(b) Under which conditions the Shapley operator of a definable game is definable in the same 
o-minimal structure? 

In the next subsection we answer the first question in a satisfactory way: if a Shapley operator 
is definable, then v n and v\ converge, to the same limit. The second question is more complex 
and will be partially answered in Section [5] 

3.2 Games with definable Shapley operator have a uniform value 

Let O be an o-minimal structure and d a positive integer. We recall the following definition: a 
subset /C C M. d is a called a cone if it satisfies R + /C C /C. 

Let || • || be a norm on K d A mapping ^ : A C K d — > K d is called nonexpansive if 

\Mf)-y(g)\\^\\f-g\\, 

whenever /, g are in M. d . Let us recall that the Shapley operator of a stochastic game is non- 
expansive with respect to the supremum norm (see |42j), norm which is defined as usual by 
H/lloo = max{/j : i = 1, ... ,d}. 

The following abstract result is strongly motivated by the operator approach to stochastic 
games, i.e. the approach in terms of Shapley operator (see Sorin [43]). It grounds on the work 
of Bewley-Kohlberg |3] and on its refinement by Neyman |30[ Th. 4], who showed that the 
convergence of the iterate *$> n (0)/n as n — > oo is guaranteed if the map A — > v \ has bounded 
variation, and deduced part (i) of the following theorem in the specific case of a semi-algebraic 
operator (301 Th. 5]. 

Theorem 3 (Nonexpansive definable mappings). The vector space M. d is endowed with an arbi- 
trary norm || • ||. Let K be a nonempty definable closed cone ofM. d and ^ : JC — > K. a definable 
nonexpansive mapping. Then 

(i) There exists v in K, such that for all f in K, the sequence ^^ n {f) converges to v as n goes 
to infinity. 

(ii) When in addition ^ is definable in a polynomially bounded structure there exists 9 G]0, 1[ 
and c > such that 

„*"(/) c 11/11 

— -v + — , 

n n a n 

for all f in fC. 

Proof. For any A € (0, 1], we can apply Banach fixed point theorem to define V\ as the unique 
fixed point of the map ^((l — A) •) and set v\ = \V\ (recall that K, is a cone). The graph of 
V\ is given by {(A, /) 6 (0, 1] X K, : ^((1 - A)/) - / = 0}. Using Proposition [2j we obtain that 
A — > V\ and A — > v\ are definable in O. Observe also that 

\\v x \\ = \\y((i-\)v x )\\ 

< ||*((l-A)Vx)-*(0)|| + ||*(0)|| 
^ ||(1 -\)V X \\ + ||*(0)|| 
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so that the curve A — > v\ is bounded by ||^(0)||. Applying the monotonicity lemma to each 
component of this curve, we obtain that v\ is piecewise C , has a limit as A goes to which we 
denote by v = vq. In order to establish that 



1 d 

'I— v x \\dX < +00, (3.10) 
10 dX 

we first observe that there exists a constant a > such that 

1 M d „ „ Z" 1 „ d 

\\—vx\\dX^a W—vxWoodX. 
«A J Q dX 

It suffices thus to establish that (|3.10p holds for the specific case of the supremum norm. Applying 
the monotonicity lemma to the coordinate functions of V\, we obtain the existence of e € (0, 1) and 
of an index iq € {1, . . . , d} such that v\ is in C 1 (0, e) with, for all A € (0, e), | (1^)10 1 = ||^^a||oo 
and (v[)i is of constant sign. Assume for instance that A — > (v' x )i is positive on (0, e). This 
shows that 

d f e 

\ — v A ||oo dX = J (v' x ) io dX = (v e ) io - 0o)i , 

and (I3~T01) follows. 

Let A such that A — > v\ is C 1 on (0, A). Let A > /x be in (0,A). Then for any decreasing 
sequence (Ai)igN in (A,//), we have 

+00 A d 

Indeed ||i>A i+1 ~ v \ II ^ II /a * 1 HX^^H ^ fx* 1 1 1 3X^11*^' so ^ na ^ ^he resu lt follows by summation. 

The map A — > v a is thus of bounded variation, and (i) follows from Neyman's proof that 
the latter property implies the convergence of ^ n (0)/n to the limit v : = limA_;.o+ V X EH]- Some 
intermediary results in Neyman's proof are necessary to establish the rate of convergence of (ii); 
we thus include the the remaining part of the proof of (i). First observe that 

I^(0)|| V/G/C (3.12) 

n n n 

for all positive integers n, so it suffices to establish the convergence result for / = 0. 
For n in N, define 

d n := ||m; 1/n -tf n (0)||HI^/n-^(0)||, 
and let us prove that n~ 1 d n tends to zero as n goes to infinity. If n > 0, we have 

d n = \M(n-l)v 1/n )-* n (0)\\ 

^ d n -i + (n - l)\\v x / n - (3-13) 

Let 
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Using (|3.13p and a discrete integration by parts, we get 



n-1 



d n ^ - vi/i\\ +d x (3.14) 

n—1 n—1 

= £ »(A - A+i) + di = X) A - (n - 1) A + d 1 . (3.15) 

i=l i=l 

Since D n tends to as n — > oo, the Cesaro sum n _1 X]£i A a ls° tends to as n — > oo. Then, 
it follows from fj3. 15[) that ra _1 d n tends to as n — >• oo. 

Finally ||« 1/n - £^ n (0)|| tends to as n goes to infinity. We know from the monotonicity 
lemma (or from the fact that v\ as bounded variation) that V\i n converges to some v. It follows 
that — ^"(0) also converges to v. 

We now prove (ii). Assume that ^ is definable in a polynomially bounded structure and 
recall that the monotonicity lemma implies the existence of A £ (0, 1) such that v is C 1 on the 
open interval (0, A) and continuous on [0, A). Since O is a polynomially bounded structure and 
since the first derivative of v is also definable in O (see Proposition [3]) , there exist 7 and c\ ^ 
such that H^t^aII = C1A -7 + o(A~ 7 ) (see Corollary [1]) . If we are able to deal with the case when 
7 is positive, the other case follow trivially. Assume thus that 7 is positive; note that, since 4rV\ 
is integrable, we must also have 7 < 1. Let C2 > be such that 

d\ 

for all positive A small enough. Let us now consider a positive integer i which is sufficiently large; 
by using (|3.1ip . we have 



i\\ v i/i - v l/i+i\\ < i I i \\~[\ Vx \\ dX ( 3 - 16 ) 

L 

^ i I C2\~ 1 d\ 

I+T 

I ' c 2 X~ 1 \" i dX 

i+T 



C2 
C2 



1 

i+1 



■I-7 

-«i + iy-p) (3.17) 



1 + 7 

Replacing c 2 by a bigger constant, we may actually assume that (|3.17p holds for all positive 
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integers. Hence 

# n (0) 



mi = n 

n n 

i=l 



1 d n < n 1 ^Jiljui/i+i - v 1 / i \\ - n 1 di 



n 

C2 



< n^Y — —(P-ii + l^-n- 1 ^ 



n 

Recalling the estimate (|3. 12[) and observing that 



C2 (n + iyy _! 

^ n di 

1 + 7 n 



f n (0) „* n (0) 

v \\ ^ v ± + K'i ~~ u 



n n 



l /"» l l 

the conclusion follows by setting 9 = 1 — 7 [0 € (0, 1)). □ 

The above result and some of its consequences can be recast within game theory as follows. 
Point (iii) of the following corollary is essentially due to Mertens-Neymann |25] , 

Corollary 4. If the Shapley operator of a stochastic game is definable the following assertions 
hold true. 

(i) The limits of v\ and v n coincide, i.e. 

lim v n = lim v\ := Woo- 

(ii) If & is definable in a polynomially bounded o-minimal structure, there exists 9 £ (0, 1] such 
that 

IK - v oo\\ = 0{—s). 

n° 

(iii) (Mertens-Neyman, |25j) The game has a uniform value. 



Proof. Since the Shapley Operator of a game is nonexpansive for the supremum norm, the two 
first points are a mere rephrasing of the proof of Theorem |3j Concerning the last one, we note 
from the proof (see (|3.10p ). that there exists an L 1 definable function <f> : (0, 1) — > R+ such that 

||«a-u m ||< f <t>{s)ds, (3.18) 

whenever A < fi are in (0, 1). Applying |25[ Theorem of p. 54], the result follows^. □ 

3 In |25] the authors uniquely consider finite stochastic games, however their proof relies only on the prop- 
erty ()3.18[) . We are indebted to X. Venel for his valuable advices on this aspect. 
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Remark 4. The first two items of Corollary [4] remain true if we do not assume that players 
observe their stage payoffs (since the value v\ does not depend on this observation). 

Remark 5. In the particular case of finite games, more is known: it is proved in [29j that the 
real 6 in (ii) can be chosen depending only on the dimension (number of states and actions) of 
the game. 

Remark 6. The definability of the Shapley operator and its Lipschitz continuity imply by [8l 
Theorem 1] its semi-smoothness. This suggests that the use of nonsmooth Newton's method for 
the computation of the values of discounted games should yield good local behaviors |16| Section 



4 Definability of the value function for parametric games 

Let O be an o-minimal structure over R. The previous section showed the importance of proving 
the definability of the Shapley operator of a game. 

Recall that the Shapley operator associates to each vector / in R d , the values of d zero-sum 
games 



where k ranges over {1, . . . ,d}. Hence each coordinate function of the operator can be seen as 
the value of a static zero-sum game depending on a vector parameter /. In this section we thus 
turn our attention to the analysis of parametric zero-sum games with definable data. 

Consider nonempty compact sets X CW,Y cM. q , an arbitrary nonempty set Z C R rf and a 
continuous pay-off function g:Ix7xZ-*l, The sets X and Y are action spaces for players 
1 and 2, whereas Z is a parameter space. Denote by A(X) (resp. A(Y)) the set of probability 
measures over X (resp. Y). When z £ Z is fixed, the mixed extension of g over A(X) x A(Y) 
defines a zero-sum game T(z) whose value is denoted by V(z) (recall that the max and min 
commutes by Sion's theorem): 



In the sequel a parametric zero-sum game is denoted by (X,Y, Z, g); when the objects 
X, Y, Z, g are definable, the game (X, Y, Z, g) is called definable. 

The issue we would like to address in this section is: can we assert that the value function 
V : Z — > R is definable in O whenever the game (X, Y, Z, g) is definable in Ol 

As shown in a forthcoming section, the answer to the previous question is not positive in 
general; but as we shall see additional algebraic or geometric structure may ensure the definability 
of the value function. 

4.1 Separable parametric games 

The following type of games and the ideas of convexification used in their studies seems to 
originate in the work of Dresher-Karlin-Shapley |12j (where these games appear as polynomial- 
like games). 



3.3]. 



max mm / / 

tieA(x) u&A(Y) Jx Jy 





(4.1) 



(4.2) 
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When x±, . . . , x m are vectors in M. p , the convex envelope of the family {x±, . . . , x m } is denoted 

by 

C0{X!,... ,X m }. 

Definition 3 (Separable functions and games). Let X C M p , Y C K 9 , Z C M d and 

j:IxFxZ-)-Rbeas above. 

(i) The function g is called separable with respect to the variables x, y, if it is of the form 

/ J 

g{x,y,z) = ^2^2mij(z)ai(x, z)bj(y, z). 

i=l j=l 

where I, J are positive integers and the continuous functions. 

The function g is called separably definable, if in addition the functions definable. 

(ii) A parametric game {X, Y, Z, g) is called separably definable, if its payoff function g is itself 
separably definable. 

Proposition 4 (Separable definable parametric games). Let (X, Y, Z,g) be a separably definable 
zero-sum game. Then the value function Z B z —¥ V{z) is definable in O. 

Proof. Let us consider the correspondence C : Z z4 M. defined by 

C(z) = co{(a\(x, z), ■ ■ ■ , ai(x, z)) : x £ X} 

and define A4 : Z z4 R* 7 similarly by Ai(z) = co{(&i(y, 2), • • • ,bj(y,z)) : y € X}. Using 
Caratheodory's theorem, we observe that the graph of C is defined by a first order formula, as 
(z, s) € Z x R 1 if and only if 

m i+i 

3(Ai, . . . , A/+i) € 3(xi, . . . , zj+i) € ^ Aj = 1, s = W^, z) . 

i=l i=l 

This ensures the definability of £ and A4. Let us introduce the definable matrix- valued function 

Z 3 z ^ M(z) = [mij(z)]i^i tl ^j 

and the mapping 

W(z) = sup inf 5M(z)T*. 

Using again Proposition [2] we obtain easily that W is definable. Let us prove that W = V, 
which will conclude the proof. Using the linearity of the integral 

I J 

W(z) = sup inf SM(z)T t = sup inf N N rriij(z)Sibj(y, z) 
SeC(z)TeM(z) SeC^y^Yf^f^ 



^ sup inf / g(x,y,z)d/i 
ueA(x) v & Jx 



= V(z). 

An analogous inequality for inf sup and a minmax argument imply the result. □ 
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4.2 Definable parametric games with convex payoff 

Scalar products on R m spaces are denoted by (•,•). 
We consider parametric games (X,Y, Z,g) such that: 

( y >• M 

Y and the partial payoff q r , : < , . are both convex. (4.3) 

p y y ' I y -»• g{x,y,z) y ' 

One could alternatively assume that X is convex and that player 1 is facing a concave function 
g VtZ for each y, z fixed. 

We recall some well-known concepts of convex analysis (see [M]). If / : ^ p — > ( — oo, +oo] is 
a convex function its subdifferential df(x) at x is defined by 

x* e 8 f{x) & f(y) > f{x) + (x*,y- x), Vy e M p , 

whenever fix) is finite; else we set 8 fix) = 0. When C is a closed convex set and x € C, the 
normal cone to C at x is given by 

iVcOc) := {uef: (v,y-x) s^0,Vy€C}. 

The indicator function of C, written Ic, is defined by Ic(x) = if x is in C, Ici%) = +oo 
otherwise. It is straightforward to see that die = Nc (where we adopt the convention iVc(x) = 
whenever x ^ C). 

Proposition 5. Let (X, Y, Z, g) be a zero-sum parametric game. Assume that Y and g satisfy 
(Ijl. Then 



(i) The value Viz) of the game coincides with 

max min hgjxj, y, z), 

ix!,...,x q+1 ) eX q+l y£Y i=1 
A € A q+1 

where A q+ i = {(Ai, . . . , X q +i) € : Y^i=i A« = 1} denotes the q + 1 simplex. 

(ii) If the payoff function g is definable then so is the value mapping V . 

Proof. Item (ii) follows from the fact that (i) provides a first order formula that describes the 
graph of V . 

Let us establish (i). In what follows d systematically denotes the subdifferentiation with 
respect to the variable y EY, the other variables being fixed. 

Fix z in the parameter space. Let us introduce the following continuous function 

$iy,z) = msxgix,y,z). (4.4) 

$(•, z) is clearly convex and continuous. Let us denote by y a minimizer of $(•, z) over Y. Using 
the sum rule for the subdifferential of convex functions, we obtain 

8$iy,z) + N Y iy) 3 0. (4.5) 
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Now from the envelope's theorem (see |34|). we know that d$(y,z) = co{dg(x, y, z) : x G 
J(y,z)}, where J(y,z) := {x in X which maximizes g(x,y,z) over X}. Hence Caratheodory's 
theorem implies the existence of fi G x±,..., x g +i G X such that 

9+1 

^^(x^Vj+iVyi^O. (4.6) 
i=l 

where, for each i, £j is a maximizer of x — > g(x, y, z) over the compact set X. Being given x in 
X, the Dirac measure at x is denoted by S x . We now establish that x, = X^=i tH^Xi and y are 
optimal strategies in the game T(z). Let x be in X, we have 

/ g(x,y)dx = y2(j,ig(xi,y,z) (4.7) 

i 

= g(xi,y,z) 
> g(x,y,z)- 



Using the sum rule for the subdifferential, we see that (|4.6p rewrites 



9 (Hg(xi, z) + Iy^j (y) 3 0, 



where Jy denotes the indicator function of Y. The above equation implies that y is a minimizer 
of the convex function fj,ig(xi, •, z) over Y . This implies that 



/ g{x,y)dx = y2mg(xi,y,z) 
Jx y 

< ^2^ig{xi,y,z)- 



Together with (|4.7p . this shows that (x,y) is a saddle point of the mixed extension of g with 
value f x g(x,y)dx. To conclude, we finally observe that we also have 

9+1 9+1 
*Y^Hig{xi,y,z) = g(xi,y,z) ^ ^ A^x;, y, z) 
t=l t=i 

for all A G A^+i and Xj in X. Hence ((A, x±, . . . , Xg+i), y) is a saddle point of the map 
(X,xi, . . .,x q+ i),y) -+ 2'=i hg{xiiViZ) with value ]P Vi9(xi,y, z) = f x g(x,y)dx. □ 

Remark 7. (a) Observe that the above proof actually yields optimal strategies for both players, 
(b) An analogous result holds, when we assume that X is convex and X B x — > g(x,y,p) is a 
concave function. 

4.3 A semi-algebraic parametric game whose value function is not semi- 
algebraic 

The following lemma is adapted from an example, due to McKinsey |24| Ex. 10.12 p 204], of 
a one-shot game played on the square where the payoff is a rational function yet the value is 
transcendental. 
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Lemma 6. Consider the semi-algebraic payoff function 

(l + x)(l + yz) 

q(x,y,z) = ; — — 

yy y ' 2(1 + xy) 2 

where (x,y,z) evolves in [0,1] x [0,1] x (0,1]. Then 

Proof. Fix z in (0, 1]. Player 1 can guarantee V(z) by playing the probability density 

dx 



ln(l + z)(l+x) 

on [0, z] since for any y E [0,1], 

g(x,y,z)dx \ + yz f z dx 



/o ln(l + z)(l + x) 21n(l + z)7 (1 + xy) 2 21n(l + z) 
On the other hand, Player 2 can guarantee V{z) by playing the probability density 

zdy 

]a(l + z)(l + yz) 

on [0, 1] since for any x G [0, 1], 

nl zg(x,y,z)dy z(l + x) f 1 dy z 



ln(l + «)(l+yz) 21n(l + z)7 (1 + xy) 2 21n(l + z 



□ 



We see on this example that the underlying objects of the initial game are semi-algebraic 
while the value function is not. Observe however that the value function is definable in a larger 
structure since it is globally subanalytic (the log function only appears through its restriction on 
compact sets). The question of the possible definability of the value function in a larger structure 
is exciting but it seems difficult, it is certainly a matter for future research. 



5 Values of stochastic games 
5.1 Definable stochastic games 

We start by a simple result. Recall that a stochastic game has perfect information if each state 
is controlled by only one of the players (see Section 13. ip . 

Proposition 7 (Definable games with perfect information). Games with perfect information and 
bounded payoj^ have a uniform value. 

Proof. Let ojf. be any state controlled by the first player. The Shapley operator in this state can 
be written as 

d 

g(x,uj k ) + ^2p(ui\x,uj k )fi . 
i=l 

So is the supremum, taken on a definable set, of definable functions, and is thus definable 
(see Example [T]). The same is true if u k is controlled by the second player, so we conclude by 
Corollary S) □ 

recall that we do not need to assume continuity of g and p in that stated in Remark [3] 



= sup 

X 
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A stochastic game (fi, X, Y, g, p) is called separably definable, if both the payoff and the 
transition functions are separably definable. More precisely: 

(a) U is finite and X C W, Y C M g are definable sets. 

(b) For each state oj, the reward function g(-, -,oj) has a definable/separable structure, that is 

lid J LO 

g(x,y,uj) := ^ ^ mf tj a^x, u) bj(y, lo), V(x,y) £1x7, 

i=l j=\ 

where I w , J w are positive integers, mfj are real numbers, ai(-,oo) and bj(-,oo) are continuous 
definable functions. 

(c) For each couple of states 00,00', the transition function p(oo'\-, -,oo) has a definable/separable 
structure, that is 

p(oo'\x,y,uo) 

where K^.u'), £(u>,u/) are positive integers, nf^ ,ul 1 are real numbers, Cj(-, cj, a/) and dj(-,oo, 00') 
are continuous definable functions. 

The most natural example of separably definable games are games with semi-algebraic action 
spaces and polynomial reward and transition functions. 

Theorem 5 (Separably definable games). Separably definable games have a uniform value. 

Proof. The coordinate functions of the Shapley operator yield d parametric separable definable 
games. Hence the Shapley operator of the game, say ^f, is itself definable. Applying Corollary [4] 
to ty, the result follows. □ 

An important subclass of separable definable games is the class of definable games for which 
one of the player has a finite set of strategies. 

Corollary 6 (Definable games finite on one-side). Consider a definable stochastic game and 
assume that one of the player has a finite set of strategies. Then the game has a uniform value. 

Proof. It suffices to observe that the mixed extension of the game is both separable and definable, 
and to apply the previous theorem. 

One could alternatively observe that the mixed extension fulfills the convexity assumptions of 
Proposition [5j This shows that the Shapley operator of the game is definable, hence Corollary [4] 
applies and yields the result. □ 

The above theorems generalize in particular the results of Bewley-Kohlberg |4], Mertens- 
Neyman |25] on finite stochastic games. 

As shown by the following result, it is not true in general that semi-algebraic stochastic games 
have a semi-algebraic Shapley operator. 



:= XI XI n M Ci(x,uj,uj')dj(y,oj,J) V(x,y) £ X xY, 
i=i j=i 
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Proposition 8. Consider the following stochastic game with two states {wi,^} and action sets 
[0, 1] for each player. The first state is absorbing with payoff 0, while for the second state, the 
payoff is 

and the transition probability is given by 

(1 +x)y 



1 - p(wi|x,y,w 2 ) = p(u 2 \x,y,U} 2 ) 



2(1 + xy) 2 ' 



for all (x, y) in [0, l] 2 . 

This stochastic game is defined by semi- algebraic and continuous functions but neither the 
Shapley operator ^ nor the curve of values (i>A)Ae(o.i] are semi- algebraic mappings. 

Proof. Notice first that p(cd2\x, y, L02) £ [0, 1] for all x and y so the game is well defined. It is 
straightforward that f 2 ) = fi, and ^(/l, $2) = fi + V(f2 — fi) (where V is the value of 

the parametric game in Lemma [6| hence is not semi algebraic. 

For any A €]0, 1[ let u\ = ^0, A ^ e j^a "^ , the identity u\ = v\ will follow as we prove that 

u\ = X^(^-^ u\). This is clear for the first coordinate, and for the second, since u\ = 
e~^~ — 1 €]0, 1[, Lemma [6] implies that 

A^ 2 (^n A ) = XV(e^-l) 

l-A 

e 2 — 1 

□ 

Remark 8. As in Lemma [6l one observes that both the Shapley operator and the curve of 
values (va)a£(o,i] are globally subanalytic. 

5.2 Stochastic games with separable definable transitions 

Theorem 7 (Games with separable definable transitions). Let (Q, X,Y, g, p) be a stochastic 
game, and assume that: 

(i) £1 is finite and X,Y are definable, 

(ii) the reward function g is an arbitrary continuous function, 

(Hi) the transition function p is definable and separable (e.g. polynomial). 

Then the game (£l,X,Y,g, p) has a uniform value. 

As it appears below, the proof of the above theorem relies on Mertens-Neyman uniform value 
theorem [25] that we do not reproduce here. We shall however provide a complete proof of a 
weaker result in the spirit of the "asymptotic approach" of Rosenberg-Sorin: 
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Theorem 8 (Games with separable definable transitions - weak version). We consider a stochas- 
tic game (Q,X,Y,g,p) which is as in Theorem^ 
Then the following limits exist and coincide: 

lim v n = lim v\. 

n— >=oo A— i>0 

Before establishing the above results, we need some abstract results that allow to deal with 
certain approximation of stochastic games. In the following proposition, the space (X, || • ||) 
denotes a real Banach space and K, denotes a nonempty closed cone of X. Being given two 
mappings _*i, <3?2 :/-—>•/_, we define their supremum "norm" through 

Pi - $ 2 ||oo = sup {!!$!(/) - * 2 (/)|| :/€£}. 

Observe that the above value may be +00, so that || • ||oo is not longer a norm, however, 
^(^i, <3?2) := — ^2||oo/(l + ll^i ~~ 3_||oo) does provide a proper metriqf|on the space of map- 
pings /C — ?> /C We say that a sequence : K. — > K. (k G N) converges uniformly to ^ : /C — > /C if 
ll^fc — ^|| 00 tends to zero as k goes to infinity, or equivalently, if it converges to with respect 
to the metric 5. The observation that the set of nonexpansive mappings ^ : /C — > /C such that 
the limit lim.n_5.0o ^ n (0)/n does exist is closed in the topology of uniform convergence was made 
in [18]. 



Proposition 9. Let : K, — s> K, be a sequence of nonexpansive mappings. Assume that 

(i) There exists : K, — > K. such that converges uniformly to ^, 

(ii) for each fixed integer k, the sequence — ^(0) has a limit v k in K. as n — > +00. 

Then the sequence v k has a limit v in K, is nonexpansive and ^^ n (0) converges to v as k 
goes to infinity. 

Proof. Take e > 0. Note first, that if $i,<3?2 are two nonexpansive mappings such that ||<_i — 
^2||oo ^ e> we have || — $2 lloo ^ ne - This follows indeed from an induction argument. The 
result obviously holds for n = 1, so assume that n ^ 2 and consider that the inequality holds at 
n — 1. For all / in /C, we have 

n*?(/) - mm < ii*i(*? _1 c/)) - *ii*r\f))\\ + ii*i(*2 -1 c/)) - **(*r x u))\\ 
^ \m-\f)-*r x u)\\+e 

^ ne. (5.1) 

Let us now prove that v k is a Cauchy sequence. Let N > be such that || _* p — ^Hoo ^ Cj f° r 
all p,q ^ N. Then, for each p,q ^ N and each positive integer n, we have 

„^(0) ^(0) 

\\ ^ e- 

n n 

Letting n goes to infinity (p and q are fixed), one gets \\v p — v q \\ ^ e and thus v k converges to a 
vector v belonging to /C. 

Take e > 0. Let N be such that || * p - ^ e/3 and - v\\ < e/3 for all p ^ iV. Using 

(|5.ip . one obtains || _■"(()) — ^ n (0)|| ne/3 where n > is an arbitrary integer. Whence 

11 * n (0),, ^ „ _ _ *£(0) _-£(0) *«(_).. 

u ^ f — ?r + \\v F - + — 

n n n n 

3 n 



3 We of course set: 5(<J>i, $2) := 1 whenever || _ 1 — $2 
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for all n > 0. The conclusion follows by choosing n large enough. 
Similarly, we prove: 



□ 



Proposition 10. Let ^ : K, — > K, be a sequence of nonexpansive mappings. Assume that 

(i) There exists \P : /C — > K such that converges uniformly to ^ , 

(ii) for each fixed integer k, the family of fixed point v k := (^^v k ) has a limit v k in K. as 
A -> 0. 



Then the sequence v k has a limit v in K, ^> is nonexpansive and v\ := A\& 1 



to v as k goes to infinity. 

Proof. Take e > 0. Let N > be such that \\ty p 
p,q ^ N and any A G]0, 1], we have 



q\\oo 



j^v\) converges 



^ e, for all p,q ^ N. Then, for each 



IP <? I 
\ V \- V \\ 



= A 
< A 



1-A p 
1-A p 



9 r . 









r 


A 




h- 1 








A 


< ) 



+ A 



1-A p 



1-A , 



< Ac + (1-A)K-«S||. 



, P <? II ^ 

so — v x \\ ^ e. 



Letting A to 0, we get that v k is a Cauchy sequence, hence converge to some v. Moreover, 
for any p > N, 



\v ~ v x\\ ^ \\ v ~ v I + \\ v 
< 2e+||7; p -^| 



"a II + K-«aI 



for all A G]0, 1]. Hence v\ converges to v. 



□ 



Proof of Theorem^ Let k be a positive integer. From the Stone- Weierstrass theorem (see |10|). 
there exists a finite family {vrfc(-, uj); u 6 fi} of real polynomial functions 



vr fc (x,y,w) 



(5.2) 



with 8% in N*, m k Auj) in R and (x,y) inlxFcff 



such that 



supsup{|7r fc (x,y,o;) - r(x,y,u)\ : (x,y) € X x Y} < -. 

Consider now, for each positive k, the game given by (Q, X, Y, TTk,p)- Since this game is definable, 
Proposition|3]applies and the game has a value. In other words its Shapley operator ^ : M rf —} M. d 
(recall that the cardinality of is d) is such that the sequence ^^(O) has a limit as n goes to 
+oo. On the other hand, one easily sees that 

*(/)- \< + I 

whenever / is in M. d and k is positive. This proves that tyf. converges uniformly to ^. Thus by 
using Proposition and Proposition [TU] , we obtain the existence of a common limit v in W 1 of 
the sequence v n = -^ n (0) and of the family of fixed points v\. □ 
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Let us now establish the stronger version of our result. 



Hence by (15. 3p . 



Proof of Theorem [?J Let A; be a positive integer. As before we consider a finite family of real 
polynomial functions, {ir/.(-, w); uj E fi}, such that 

sup sup {\irk(x,y,Ld) — r(x,y,uj)\ : (x,y) 6 X x Y} ^ -. (5.3) 

uen k 

Consider now, for each positive k, the game T k given by X,Y, ir^,p). Since this game is 
definable, Theorem [5] applies and the game has a uniform value v k . Hence, there exists an 
integer iV (depending on k) and a strategy a of Player 1 which is -r optimal in the n-stage game 
for any n ^ N. That is, for any strategy r of Player 2 and any starting state uj, 

ln(^r,u) >v k (uj)-^. 

. 2 

j n (a,T,oj) ^ v k {uj) - -. (5.4) 

Taking the infimum over all possible strategies r, we get that for every uj and every large n, 

2 

v n (uj) ^ v k (u) - -. 

Using the dual inequality 

2 

v n (u) < w fc (w) + - (5.5) 

one gets that limsupu n (a;) — liminf v n (oj) ^ r- Hence v n converges to some v. Moreover, 
combining (|5.4p and (|5.5p yields 

4 5 

for n sufficiently large. Hence v is the uniform value of the game. □ 



An immediate consequence of Theorem [7] is the following 

Corollary 9. Any game with switching control and with a definable transition probability has a 
uniform value. 



5.3 Geometric growth in nonlinear Perron-Frobenius theory 

We finally point out an application of the present results to nonlinear Perron-Frobenius theory, 
in which Shapley operators do appear, albeit in a disguised form. 

We denote by C = M.+ the standard (closed) nonnegative cone of equipped with the 
product ordering. We are interested in maps T defined on the interior of C, satisfying some of 
the following properties. We say that T is order preserving if 

f<g=>T(f)^T(g), Vf,gemtC, 

that it is positively homogeneous (of degree 1) if 

T(Xf) = XT(f), V/eintC, VAX), 
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and positively subhomogeneous if 

T(Xf)^XT(f), V/eintC, VA> 1. 

Let log : intC — > M rf denote the map which does log entrywise, and let exp := log -1 . It is clear 
that T is order-preserving and positively homogeneous if and only if the conjugate map 

\& := log o T o exp (5-6) 

is order-preserving and commutes with the addition of a constant. These two properties hold if 
and only if ^ is a dynamic programming operator associated to an undiscounted game with state 
space {1, . . . , d}, i.e. if ^ can be written as in (|3.3p . but with possibly noncompact sets of actions 
(see in particular [22J). Note also that if T is order preserving and positively subhomogeneous, 
then, Vl' is sup-norm nonexpansive. 

In the setting of nonlinear Perron-Frobenius theory, we are interested in the existence of the 
geometric growth rate x(T), defined by 

X (T) := exp( lim n~ 1 logT n (e)) = exp( lim n^ftloge)) (5.7) 

where e is an arbitrary vector in the interior of C. 

Problems of this nature arise in population dynamics. In this context, one considers a popu- 
lation vector f(n) € intK+, where [/(n)]j represents the number of individuals of type i at time 

n, assuming a dynamics of the form f(n) = T(f(n — 1)). Then, [x(^)]« = ^ m n^oo[T n (f (O))]]^ 
represents the geometric growth rate of individuals of type i. 

Corollary 10 (Geometric Growth). LetT be an order preserving and positively subhomogeneous 
self map of int C that is definable in the log- exp structure, and let e be a vector in int C . Then, 
the growth rate x(T), defined by (|5.7p . does exist and is independent of the choice of e. 

Proof. Apply Theorem [3] to the operator (15. 6p . which is nonexpansive in the sup-norm as well as 
definable in the log-exp structure, and use (|5.7p . □ 

Here is now an application of Corollary [10] to a specific class of maps. 

Corollary 11 (Growth minimization). Assume that T is a self-map of int C every coordinate of 
which can be written as 

[T(/)]i= inf (p,f) l<i<d , (5.8) 
peMi 

where Mi is a subset ofC. Assume in addition that every Mi is definable in the log-exp structure. 
Then, the growth rate x(T) does exist and is independent of the choice of e 6 intC. 

Proof. The map T is obviously order preserving, positively homogeneous, and, by Proposition [2] 
or Example [1] it is definable in the log-exp structure as soon as every set Mi is definable in this 
structure. Hence, the result follows from Corollary [TU] □ 

Several motivations lead to consider maps of the form (|5.8p . The first motivation arises from 
discrete time controlled growth processes. As above, to each time n ^ 1 and state 1 ^ i ^ d is 
attached a population [f(n)]i. The control at time n is chosen after observing the current state 
1 ^ i ^ d. It consists in selecting a vector p G Mi. Then, the population at time i becomes 
[f(n)]i = (p,f(n — 1)). The iterate [T n (e)]j represents the minimal possible population at state 
i and time n, with an initial population e. Then, the limit x{T) represents the minimal possible 
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growth rate. This is motivated in particular by some therapeutic problems (see e.g [6]), for which 
x(T) yields a lower bound on the achievable growth rates. 

Another motivation comes from risk sensitive control |17[ |9] or from mathematical finance 
models with logarithmic utility [2]. In this context, it is useful to consider the conjugate map 
^ := log o T o exp, which has the following explicit representation 




where 

S(q,p) ■= ^2 ?i l0 S(ty'/Pj) 

denotes the relative entropy or Kullback-Leibler divergence, and := {q € C | ^i<j<d9j = 1} 
is the standard simplex. Then, \og[x{T)]i can be interpreted as the value of an ergodic risk 
sensitive problem, and it is also the value of a zero-sum game. 

The case in which inf is replaced by sup in (|5.8p . i.e., [T(/)]j = sup^g^ . (p, /}, for 1 ^ i ^ d, 
which is also of interest, turns out to be simpler. Indeed, every coordinate of the operator 
\& := log oT o exp becomes convex (this can be easily seen from the representation analogous 
to (|5.9p . in which the infimum is now replaced by a supremum). More generally, the latter 
convexity property is known to hold if and only if Vl/ is the dynamic programming operator of a 
one player stochastic game [USE]. It has been shown by several authors [T8, 48, 32j that for this 
class of operators (or games), the limit linin^+oo $/ n (f)/n does exist, from which the existence 
of the limit (|5.7p readily follows. 

Finally, we note that we may consider more general hybrid versions of (|5.8p . for instance with 
a partition {1, . . . , d} = I U J and 

[T(f)]i = inf (p, /) i El, [T(f)]i = sup (pj) i E J . 

Then the existence of the growth rate, for such maps, also follows from Corollary 1101 

Acknowledgments. The authors would like to thank X. Venel and S. Sorin for their very useful 
comments. 
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